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Abstract 

The effectiveness of four adult learning methods 
(accelerated learning, coaching, guided design, 
and just-in-time training) constituted the focus 
of this research synthesis. Findings reported in 
How People Learn (Bransford et al., 2000) were 
used to operationally define six adult learning 
method characteristics, and to code and analyze 
the relationship between the six characteristics 
and the study outcomes (learner knowledge, skills, 
attitudes, and self-efficacy beliefs). The synthesis 
included 79 studies using either randomized 
controlled trials or comparison group designs (N 
= 3,152 experimental group participants and N = 
2,988 control or comparison group participants). 
Results showed that all six adult learning method 
characteristics were associated with positive learner 
outcomes, but that learning methods and practices 
that more actively involved learners in acquiring, 
using, and evaluating new knowledge and practice 
had the most positive consequences. Results also 
showed that the adult learning methods were 
most effective when used with a small number of 
learners (< 30) for more than 10 hours on multiple 
occasions. Implications for professional training 
and technical assistance are described. 


Introduction 

The manner in which the characteristics of four differ- 
ent adult learning methods were associated with the acquisi- 
tion and mastery of new knowledge or practice is the focus 
of this research synthesis. The four methods are acceler- 
ated learning (Meier, 2000; Molnar, 2001), coaching (Har- 
greaves & Dawe, 1990; Leat, Lofthouse, & Wilcock, 2006), 
guided design (Coscarelli & White, 1986; Wales & Stager, 
1978), and just-in-time training (Davis, 2005; Rosen, 2005). 
Findings described in How People Learn (Bransford et al., 
2000; Donovan, Bransford, & Pellegrino, 1999), a research 
review on the science of learning, were used to develop cri- 
teria that were used as the standards against which the four 
adult learning methods and strategies were judged. 

The research synthesis was guided by a characteris- 
tics/consequences framework that relates variations in the 
presences of operationally-defined features of a practice to 
variations in study outcomes (Dunst & Trivette, in press; 
Dunst, Trivette, & Cutspec, 2007; Dunst, Trivette, & Wat- 
son, 2009). The focus of analysis is unpacking, disentan- 
gling, and identifying those characteristics of a practice that 
matter most in terms of explaining the relationship between 
the independent and dependent variables in studies consti- 
tuting the focus of analysis. The process is similar to that 
proposed by Lipsey (1993) for unbundling the components 
or elements of an intervention or treatment to isolate the ac- 
tive ingredients of a method or procedure. The yield is a 
better understanding of the conditions that best explain how 
and in what manner an intervention or practice exerts an ef- 
fect in one or more outcomes. 
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Background 

Adult learning refers to a collection of theories and meth- 
ods for describing the conditions under which the processes 
of learning are optimized (Merriam, 2001; Trotter, 2006; 
Yang, 2003). Knowles (1984) used the term andragogy to de- 
scribe the assumptions of adult learning: Readiness-to-learn, 
self-directedness, active learner participation, and solution- 
centered. Nearly all adult learning methods and strategies in- 
clude at least several of these elements. The four adult learn- 
ing methods constituting the focus of this research synthesis 
include, to different degrees, the characteristics that Knowles 
et al. (1998) as well as others (Trotter, 2006; Yang, 2003) 
consider the principles of adult learning. The four methods 
were selected for analysis because they all have received con- 
siderable attention and their effectiveness has been assessed 
using either randomized controlled trials or comparison group 
designs. 

Adult Learning Methods 

Each adult learning method constituting the focus of 
analysis includes similar features as well as elements unique 
to each strategy. The following are brief descriptions of the 
four adult learning methods. 

Accelerated learning. First called suggestopedia (Lo- 
zanov, 1978), this adult learning method includes procedures 
for creating a relaxed emotional state, an orchestrated and 
multi-sensory learning environment, and active learner en- 
gagement in the learning process (Meier, 2000). A relaxed 
emotional state includes relaxation and breathing exercises, 
suggestion, and a positive learning atmosphere. An orchestrat- 
ed environment includes imagery, dramatic readings, instruc- 
tional videos, and peripherals (posters and visual displays). 
Active learning includes plays or skits, role playing, practice 
exercises, group activities, and journal writing. Accelerated 
learning is considered a holistic adult learning method that is 
intended to promote creation (and not consumption), enhance 
retention, and quicken the learning process. 

Coaching. “Coaching is a. ..method of transferring 
skills and expertise from more experienced and knowledge- 
able practitioners... to less experienced ones” (Hargreaves 
& Dawe, 1990, p. 230). This adult learning method includes 
procedures for joint planning and goal setting, coach infor- 
mation sharing and modeling, learner information gathering 
and practicing, analysis of and reflection on the learner’s ex- 
periences, and coach feedback (Leat et al., 2006). Coaching 
is a learner driven process facilitated by a coach’s encourage- 
ment and use of his or her knowledge and skills to promote 
learner understanding and use of newly acquired knowledge 
and skills (Gallacher, 1997). Coaching is conceptualized as 
a cyclic process that improves knowledge and skills, self- 
confidence, and collegial relationships as a result of ongoing 
coaching episodes. 

Guided design. Guided design was developed to pro- 
mote critical thinking and self-directed learning (Hancock, 
Coscarelli, & White, 1983). This adult learning method is 
characterized by a decision-making and problem solving pro- 


cess that includes procedures for using real world problems 
for mastering learning content using small-group or team 
processing and facilitator guidance and feedback (Wales & 
Stager, 1978). The procedure was first used to teach deci- 
sion making skills to engineering students (Colvin, Kilmer, 
& Smith, 1972) but is now widely used in a number of fields 
that involve decision-making and problem solving (e.g., 
Goldberg & Shuman, 1984b; Turner & Bechtel, 1998; Yang, 
He, & Drueckhammer, 2001). The benefits of this adult learn- 
ing method include higher-order problem solving and meta- 
cognitive thinking abilities. 

Just-in-time training. Just-in-time training includes a 
number of different methods and strategies used in the con- 
text of real-life challenges and in response to learner requests 
for guidance or mentoring (Beckett, 2000). This adult learn- 
ing method provides individualized, tailored training in re- 
sponse to a request specific to an immediate concern or need 
(Redding & Kamm, 1999). According to Brandenburg and 
Ellinger (2003), just-in-time training is “often conceived as 
anywhere, anytime learning that is just enough, just for me, 
and just in time” (p. 309). The key characteristics of this adult 
learning method include access to or provision of information 
needed to improve performance or complete a task, on-the- 
job use of the information or guidance, and the availability 
of input from a mentor, supervisor, or coach on an as-needed 
basis (Bersin & O'Leonard, 2005). The primary outcome 
of just-in-time training is context specific improvement of 
knowledge and performance. 

How People Learn 

Examination of the characteristics of the four adult learn- 
ing methods finds that they share common features as well as 
have elements unique to each strategy. The extent to which the 
characteristics of the methods and strategies rather than the 
procedures per se account for learner outcomes was the focus 
of this research synthesis. This was accomplished by using the 
features of adult learning identified by Donovan et al. (1999) 
in a research review of how people acquire, learn, and master 
new material and information as benchmarks for developing 
operationally defined characteristics that were used to evalu- 
ate the effectiveness of the adult learning methods. 

The research review conducted by Donovan et al. (1999) 
identified three key elements of the “science of learning.” 
These were: (1) New material and information is more easily 
learned when it is related to existing learner knowledge and is 
relevant to the learner, (2) mastery of new material and infor- 
mation requires application of the knowledge in the context 
of a conceptual, procedural, or practical framework, and on- 
going monitoring of learning and self-assessment of progress 
facilitates deeper understanding and continued application 
of new knowledge or practice. According to Bransford et al. 
(2000), teachers, trainers, instructors, etc. play a “critical role 
in assisting learners to engage their understanding, building 
on learners’ understanding, correcting misconceptions, and 
observing and engaging with learners during the process of 
learning” (p. 238). These conditions, in various forms, define 
the key features and elements of the four adult learning meth- 
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ods constituting the focus of this research synthesis. 

The Donovan et al. (1999) findings were used to identify 
six adult learning method characteristics, two for each adult 
learning element, and to code the studies in this research syn- 
thesis using these characteristics to determine the extent to 
which the presence of the characteristics were related to vari- 
ations in study outcomes. Table 1 shows the characteristics 
that were the focus of analysis. The three main features were 
planning, application, and deep understanding. Planning in- 
cluded the methods and procedures for both: (1) Introducing 
new knowledge, material or practices and (2) illustrating and 
demonstrating the use of the knowledge, material or practices. 
Application included the methods and procedures for both: 
(1) Learner applied use of the knowledge, material or prac- 
tices and (2) the evaluation of the outcome or consequence of 
application. Deep understanding included the methods and 
procedures for: (1) Engaging the learner in reflection on his 
or her learning experience and (2) self-assessment of knowl- 
edge and application mastery as a foundation for identifying 
new learning opportunities. The six characteristics are almost 
identical to those described by Graham and Wedman (1989) 
as the key features of effective adult learning practices. 

A number of analyses were performed to identify which 
adult learning method characteristics and practices were as- 
sociated with variations in learner outcomes. First, we as- 
sessed the extent to which the different adult learning method 
characteristics were related to the study outcomes. Second, 
the particular types of practices for each adult learning meth- 
od characteristic were analyzed to determine which kinds 
of practices for each characteristic were related to the study 
outcomes. For example, six different practices were used for 
Introducing New Information, and the relative effectiveness 
of each of the practices was assessed in terms of the study 


outcomes. (The same was done for each of the adult learn- 
ing method characteristics.) Third, we determined whether 
the relationships between the adult learning method charac- 
teristics and practices differed as a function of a number of 
moderator variables (e.g., length of training). 

The main focus of analysis was the identification of 
those characteristics that could be used to inform changes 
and improvements in inservice training opportunities of adult 
learners being introduced and taught new knowledge or new 
practices. The characteristics identified as most important, 
however, have broad-based applicability to other types of 
learning or training. 

Search Strategy 

Studies that investigated the effectiveness of the four 
adult learning methods were identified by four searches, one 
for each method. Table 2 lists the search terms used to lo- 
cate studies of each adult learning method. The terms used to 
identify studies of each adult learning method are ones that 
have at different times been used interchangeably to describe 
the learning methods. The search terms were used with train, 
learn, educate, or instruct as Boolean conditions. In each of 
the searches, the term adult was also used as a Boolean con- 
dition to limit the studies to adult learners. 

Several searches had additional delimiters. For example, 
the Boolean NOT operator was used with coaching to screen 
out studies involving sports, athletics, personal trainer, and oth- 
er types of practices that use some type of a coach. Similarly, 
the NOT operator was used with just-in-time training to screen 
out studies that included inventory, management, debugger, 
and other terms that involve the use of the phrase just-in-time 
to describe practices but were not adult learning methods. 


Table 1 

Characteristics of the Adult Learning Methods That Were the Focus of Analysis 


Features/Characteristics 

Definition 

Planning 

Introduce 

Engage the learner in a preview of the material, knowledge or practice that is the focus of 
instruction or training. 

Illustrate 

Demonstrate or illustrate the use or applicability of the material, knowledge or practice for the 
learner 

Application 

Practice 

Engage the learner in the use of the material, knowledge or practice. 

Evaluate 

Engage the learner in a process of evaluating the consequence or outcome of the application of 
the material, knowledge or practice. 

Deep Understanding 

Reflection 

Engage the learner in self-assessment of his or her acquisition of knowledge and skills as a basis 
for identifying “next steps” in the learning process. 

Mastery 

Engage the learner in a process of assessing his or her experience in the context of some 
conceptual or practical model or framework, or some external set of standards or criteria. 
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Table 2 

Search Terms Used to Identify Studies of the Adult Learning Methods 


Accelerated Learning 

Coaching 

Guided Design 

Just-in-Time Training 

Accelerated Learning 
Suggestopedia 
Accelerative Learning 
Active Learning 
Hyperlearning 
Superlearning 
Brain Compatible Learning 
Brain Compatible Teaching 

Coaching 
Mentoring 
Co-Coaching 
Collaborative Training 

Guided Design 
Learning Strategies 
Participant Decision Making 
Problem Solving 

Just in Time 
Just-in-Time 
Personnel Development 
Career Development 
Employee Development 


Sources 

ERIC (Educational Resources Information Center), Psy- 
chological Abstracts (Psychlnfo), Academic Search Elite, 
Business Source Elite, World CAT, Social Sciences Cita- 
tion Index, InfoTRAC Expanded Academic ASAP, Medline, 
OCLC PapersFirst, and Dissertation Abstracts were searched 
to identify studies. These were supplemented by searches of 
Ingenta, Google Scholar, ABI/IFORM Global, the Cochrane 
Databases, and an EndNote library maintained by the Puckett 
Institute. 

Hand searches of the reference sections of all retrieved 
journal articles, book chapters, and books were examined to 
identify additional studies. Journals dedicated to the adult 
learning methods were also reviewed to identify studies (e.g.. 
Journal of Accelerated Learning and Teaching; Coaching: 
An International Journal of Theory, Research and Practice). 
Websites dedicated to the adult learning methods were also 
reviewed to identify additional studies (e.g.. International Al- 
liance for Learning; National Center for Guided Design). We 
also conducted Social Science Citation Index author searches 
of seminal papers and studies by individuals who either de- 
veloped the different adult learning methods or are leaders in 
the use of the methods to identify additional investigations. 
These individuals included, but were not limited to, Geor- 
gi Lozanov and David Meier (accelerated learning), Joyce 
Showers and Frank Kohler (coaching), Samuel Colvin and 
Charles Wales (guided design), and DeLayne Hudspeth and 
Laura Dorsey (just-in-time training). 

Inclusion Criteria 

Studies were included if the: (1) Participants were adult 
learners (defined as post high school age), (2) sufficient in- 
formation was included to code the use of the different adult 
learning method characteristics, (3) the adult learning meth- 
od was compared to some control or contrasting condition, 
and (4) either a randomized controlled trial or comparison 
group design was used to evaluate the effectiveness of the 
adult learning methods. 

Exclusion criteria. Studies were excluded if the partici- 
pants were elementary or secondary school students, insuf- 


ficient information was included about specific elements of 
the adult learning procedures, and pre-experimental or single 
participant research designs were used. Single participant de- 
sign studies were excluded because they were only used for 
evaluating the effectiveness of coaching. A research synthesis 
of coaching that includes both group and single participant 
design studies is in the process of being completed. 

Search Results 

Seventy-nine (79) studies were located in 66 research re- 
ports. An investigation was considered a separate study in a 
single research report if two or more intervention groups in 
any one investigation examined the effects of different adult 
learning method characteristics. Thirty-two (32) studies inves- 
tigated accelerated learning, 23 investigated coaching, 16 in- 
vestigated guided design, and 8 investigated just-in-time train- 
ing. Fifty -eight (58) of the studies used randomized control de- 
signs and 2 1 used non-equivalent comparison group designs. 

Study Participants 

Appendix A shows the study participants (learners), the 
settings in which the adult learning methods were imple- 
mented, and the knowledge or skill areas that were the focus 
of instruction or training. The 79 studies included 3,152 ex- 
perimental group participants and 2,988 control or compari- 
son group participants. 

The learners included classroom teachers, student teach- 
ers, undergraduate students, graduate students, medical per- 
sonnel, counselors, English-as-a-second-language learners, 
and business personnel (e.g., sales and customer service 
personnel). The settings in which the adult learning methods 
were implemented included college classrooms; elementary, 
junior and high schools; special education classrooms; hos- 
pitals and private physician practices; and various business 
and work settings. The learner outcomes in the studies in- 
cluded teaching practices, foreign language learning, nursing 
and medical practices, science and engineering, mathematics 
and statistics, economics, and rare vocabulary, among other 
outcomes. 
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Adult Learning Method Characteristics 

Appendix B shows the length of training and the particu- 
lar practices coded for each of the six adult learning method 
characteristics. The length of training ranged from one to 
over 100 hours. One third of the studies evaluated training 
provided between 1 and 10 hours, one third of the studies 
evaluated training between 11 and 40 hours, and one third 
of the studies evaluated training of more than 40 hours. The 
largest majority of studies (85%) provided training on mul- 
tiple occasions, although the exact number of sessions was 
generally not reported. 

Coding of the adult learning method characteristics 
showed that 76 studies included the introduction of some type 
of knowledge, material or practices, and 37 studies included 
the demonstration or illustration of some type of knowledge, 
material or practice. Fifty-eight (58) studies included some 
type of learner application, and 31 studies included some 
type of learner evaluation of their use of the knowledge, ma- 
terial or practices. Thirty-three (33) studies included some 
type of learner reflection, and 29 studies included some type 
of learner self-assessment of mastery. 

Planning. Sixteen (16) different practices were used to 
introduce new knowledge or practices to the study partici- 
pants which were subsequently grouped into six categories: 

(1) Class lectures, (2) warm-up exercises and pre-class quiz- 
zes, (3) self-instruction and out-of-class activities, (4) dra- 
matic readings, (5) imagery, and (6) a combination of dra- 
matic readings and imagery. There were 15 different meth- 
ods used to illustrate or demonstrate application which were 
subsequently grouped into four categories: (1) Real life dem- 
onstrations, and real life demonstrations and role playing, 

(2) role playing (simulations, skits, plays), (3) instructional 
videos, and (4) learner informed lecture content. The latter 
included instructor incorporation of learner experiences into 
class lectures or the inclusion of results from pre-class exer- 
cises for illustrating the targeted content. 

Application. Sixteen (16) different practices were used 
to engage the learners in the application of newly acquired 
information or material which were subsequently grouped 
into five categories: (1) Real life use of the knowledge or 
practice, (2) role playing (simulations, skits, plays), (3) real 
life demonstrations and role playing, (4) problem solving ac- 
tivities, and (5) games/writing exercises. Eight (8) different 
methods were used to have learners evaluate the consequenc- 
es of application which were grouped into two categories: (1) 
Instructor assessment, review, and feedback on the learners’ 
application and (2) learner review and self-assessment. The 
latter included either individual or group review and assess- 
ment of learner use of the targeted information, material, or 
practice. 

Understanding. Four (4) different methods were used 
to engage learners in reflection on knowledge acquisition 
and practice application which were organized into three 
categories: (1) Performance improvement reviews, (2) jour- 
naling and behavioral suggestions, and (3) group reflection 
on instructor feedback. Performance improvement reviews 
involved joint learner and instructor discussions of learner 


application for the purpose of deciding “next steps” in the 
learning process. Journaling and behavioral suggestions in- 
volved strategies for engaging learners in self-reflection on 
their learning as a way of focusing their attention to “next 
steps.” Group reflection involved learner processing of in- 
structor feedback on application to promote deeper under- 
standing of the learning topic. 

Learner mastery was determined by: (1) self-assessment 
of personal strengths (and weaknesses) and (2) evaluation 
of learner performance against a set of standards or prac- 
tice criteria. Both methods involved the same type of learner 
judgments of their own knowledge and performance in the 
context of some framework against which application was 
assessed. Self-assessment of learner strengths and weakness- 
es was done either individually or as a group in response to 
instructor feedback as a basis for self-assessing learner mas- 
tery. Learner assessment of mastery used a priori established 
standards or competencies as criteria against which learner 
knowledge and performance was assessed. 

Study Outcomes 

The outcomes listed in Appendix A (and also Appendix 
C below) were organized into four categories: (1) Knowl- 
edge, (2) skills, (3) learner attitudes, and (4) self-efficacy 
beliefs. Knowledge included learner mastery of course con- 
tent, second language learning, memorization, job require- 
ments, medical procedures, and other content. Skills included 
learner teaching methods, computer use, medical procedures, 
interviewing skills, job performance, and practitioner inter- 
vention capabilities. Learner attitudes included evaluation 
of the learning experience and satisfaction with the learning 
procedures. Class attendance and completion of the learner 
intervention were used as proxy measures of learner attitudes. 
Self-efficacy included learner judgments of their competence 
and confidence in their own abilities and perceived control in 
the ability to produce intended outcomes. Both standardiza- 
tion and investigator-developed measures were used to assess 
learner outcomes. 

Synthesis Findings 

Cohen’s d effect sizes for the mean difference in the 
post-test study outcomes between the experimental groups 
and the control or comparison groups were used for assessing 
the effectiveness of the adult learning methods. The average 
Cohen’s d was computed for each of the six adult learning 
method characteristics as well as type of practice for each 
characteristic to ascertain which characteristics and practices 
accounted for the largest between group differences. The av- 
erage sizes of effect and the 95% confidence intervals for the 
mean effect sizes were used for substantive interpretation. A 
confidence interval not including zero indicates that the av- 
erage effect size is significantly greater than zero at the .05 
level (Hedges, 1994). 

Appendix C includes the complete list of studies, the 
research designs, the comparisons that were made, the out- 
comes that constituted the focus of analysis, and the Cohen’s 
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d for each outcome measure. The average effect size and 95% 
confidence interval (Cl) for all studies and outcomes com- 
bined was .58 (Cl = .45-. 70). Each of the four adult learning 
methods were associated with positive post-test outcome dif- 
ferences between the experimental and control or comparison 
groups. The average effects sizes and 95% confidence inter- 
vals for the individual adult learning methods was .86 (Cl 
= .41-1.31) for just-in-time training, .68 (Cl = .47-. 90) for 
coaching, .67 (Cl = .39-. 95) for guided design, and .35 (Cl = 
.20-.51) for accelerated learning. 

Omnibus Findings 

Figure 1 shows the average effect sizes and 95% con- 
fidence intervals for each of the six adult learning method 
characteristics. What is shown is the average sizes of effects 
for all practice characteristics combined and for all outcomes 
measures combined. 

Each of the adult learning method characteristics was 
moderately to highly related to the study outcomes. The av- 
erage effect sizes for the six adult learning method character- 
istics ranged between .58 and .85, which indicates that there 
was one-half to one standard deviation difference between the 
experimental and control or comparison groups on the post- 
test outcome measures. The pattern of results show that the 
more actively involved the learners were in judging the con- 
sequences of their learning experiences (evaluate, reflection, 
& mastery), the stronger the relationship between the adult 
learning method characteristics and the study outcomes. The 
latter indicates the relative importance of active learner par- 
ticipation in learning new knowledge or practice, and learner 
engagement in judging his or her experience in learning and 
using new material. 

Adult Learning Method Characteristics 

Table 3 shows the results for the different practices used 
for each adult learning method characteristic. All of the meth- 
ods, except a combination of imagery and dramatic readings 
for introducing new information, were significantly related to 
the study outcomes. The effects of the different adult learning 
method practices, however, were not the same as evidenced 
by the variability in the average effect sizes and 95% confi- 
dence intervals for the different practices for each adult learn- 
ing method characteristic. 

Introduction of the learning topic. Two methods proved 
to be the most effective in terms of introducing new knowl- 
edge, material or practices to the learners: (1) Out-of-class 
activities and self-instruction and (2) warm-up exercises and 
pre-class quizzes. Both practices actively involved learners in 
some type of exercise or task prior to instructor presentation 
or explanation of the learning topic as a means of introducing 
new information or practice. Classroom lectures were also 
significantly related to the study outcomes but not as strongly 
as active learner participation in having them introduced to 
the learning topic. Dramatic readings and imagery, either 
alone or in combination, had considerably smaller effects or 
no effect on learner outcomes. 

Illustrating the learning topic. Two methods for illustrat- 
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ADULT LEARNING METHOD CHARACTERISTICS 

Figure 1. Average effect size and 95% confidence inter- 
vals for the relationship between the six adult learning 
method characteristics and the study outcomes. 

ing the use of new knowledge, material or practices were most 
strongly related to the study outcomes: (1) Instructor use of 
role playing or simulations and (2) incorporating learner input 
into demonstrating the applicability of the new knowledge, 
material or practices. A combination of real life demonstra- 
tions and role plays also proved to be an effective strategy for 
illustrating the learning topic. Passively watching an instruc- 
tional video was effective but not nearly as effective as strate- 
gies more actively involving the learners in the illustrations. 

Practicing the use of the learning topic. A combination 
of real life application and role plays proved to be the most 
effective method for engaging learners in the use of the new- 
ly-learned knowledge, material or practice. Problem solving 
tasks, real life application, and some type of learning game 
or writing exercises also proved to be highly effective for 
engaging learners in application. Role plays, simulations or 
skits were also significantly related to the study outcomes but 
not as strongly as the other methods for engaging learners in 
application. 

Evaluating the consequences of application. The two 
methods for engaging learners in the evaluation of the conse- 
quences of their use of the new knowledge, material or prac- 
tices proved equally effective: ( 1 ) Assessing learner strengths 
and weaknesses related to the application experience and (2) 
reviewing learner solutions to problems or answers to quiz- 
zes about their experiences. 

Reflection on learner acquisition. Engaging the learner 
in a process of determining the next steps in learning targeted 
knowledge or practice was most effective (performance im- 
provement). Engaging learners in journaling about their new- 
ly acquired knowledge and skills or positive learner feedback 
were also effective strategies for learner reflection. Group 
reflection on instructor feedback or peer feedback was an ef- 
fective method of reflection as well. 

Learner assessment of mastery. Actively involving learn- 
ers in some type of self-assessment of their mastery of the 
learning topic or having learners use a set of standards or ex- 
ternal criteria for assessing their learning were both strongly 
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Table 3 

Effect Sizes for the Different Adult Learning Method Characteristics and Practices 


Characteristics / Practices 

Number 

Studies Effect Sizes 

Mean Effect 
(d ) Size 

95% Confidence 
Interval 

Introducing Information 

Pre-class exercises 

9 

9 

1.02 

.63-1.41 

Out of class activities/self-instruction 

12 

20 

.76 

.44-1.09 

Classroom/workshop lectures 

26 

108 

.68 

.47-. 89 

Dramatic readings 

18 

40 

.35 

.13-. 57 

Imagery 

7 

18 

.34 

.08-. 59 

Dramatic readings/imagery 

4 

11 

.15 

-.33-. 62 

Illustrate/Demonstrate 

Learner input 

6 

6 

.89 

.28-1.51 

Role playing/simulation 

20 

64 

.87 

.58-1.17 

Real life example/real life + roleplaying 

6 

10 

.67 

.27-1.07 

Instructional video 

5 

49 

.33 

.09-. 59 

Practicing 

Real life application + role playing 

5 

20 

1.10 

.48-1.72 

Problem solving tasks 

16 

29 

.67 

.39-. 95 

Real life application 

17 

83 

.58 

.35-. 81 

Learning games/writing exercises 

9 

11 

.55 

.11-. 99 

Role playing (skits, plays) 

11 

35 

.41 

.21-. 62 

Evaluation 

Assess strengths/weaknesses 

14 

48 

.96 

.67-1.26 

Review experience/make changes 

19 

35 

.60 

.36-. 83 

Reflection 

Performance improvement 

9 

34 

1.07 

.69-1.45 

Journaling/behavior suggestion 

8 

17 

.75 

.49-1.00 

Group discussion about feedback 

16 

29 

.67 

.39-. 95 

Mastery 

Standards-based assessment 

13 

44 

.76 

.42-1.10 

Self-assessment 

16 

29 

.67 

.39-. 95 


related to the study outcomes for assessing their performance. 

Learner Outcomes 

The average effect size between the adult learning meth- 
ods (taken together) and all outcomes measures combined 
was d = .58, Cl = .45-. 70. The influences of the adult learning 
method characteristics on the four different outcome mea- 
sures found that learner skill acquisition (d = .66, Cl = .43- 
.88) and learner attitudes were most strongly related to the 
practices (d = .68, Cl = .40-. 96), followed by learner knowl- 
edge (d = .49, Cl = .29-. 69) and learner self-efficacy beliefs 
(d = .47, Cl = .28-. 65). 

Figure 2 shows the average sizes of effects and 95% con- 
fidence intervals for the relationships between the six adult 
learning method characteristics and the four types of learner 
outcomes. In every analysis except one (learner evaluation 
and self-efficacy beliefs), the different adult learning method 
characteristics were significantly related to the measures in 
the four categories of study outcomes. Close inspection of 



Introduce Illustrate Practice Evaluate Reflection Mastery 


ADULT LEARNING METHOD CHARACTERISTICS 


Figure 2. Average effect sizes and 95% confidence inter- 
vals for the relationships between the six adult learning 
method characteristics and four categories of study out- 
comes. 
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the findings, however, shows that the different adult learn- 
ing method characteristics were differently related to the four 
study outcomes. 

In almost every set of analyses, the six adult learning 
method characteristics were more strongly related to learner 
knowledge, skills, and attitudes compared to learner self- 
efficacy beliefs. Instructor illustration or demonstration of 
the targeted knowledge or practice was most strongly related 
to learner attitudes, whereas both learner evaluation of and 
reflection on the targeted knowledge or practice was more 
strongly related to learner skill acquisition compared to the 
other adult learning method characteristics. 

Combined Influences of the Adult 
Learning Method Characteristics 

The extent to which the simultaneous presence or use of 
the different adult learning method characteristics was dif- 
ferentially related to the study outcomes was determined by 
summing the number of characteristics per study and relating 
variations in this measure to learner outcomes. The presence 
of a characteristic was limited to only those practices that 
had effect sizes of .66 or higher (Table 3) in order to include 
those practices that were most strongly associated with posi- 
tive learner consequences. The average number of character- 
istics per study was 2.18 (SD = 1.63, Range = 0 to 5) using 
this metric. 

Figure 3 shows the results for the relationship between 
the number of practice characteristics and the learner out- 
comes. The patterns of findings are both clear and illuminat- 
ing. The more adult learning method characteristics that were 
used, the larger the sizes of effects between the practices and 
the study outcomes. Studies where none or only one charac- 
teristic was used had little or no effect on learner outcomes. 
Studies that included 2, 3, or 4 characteristics were associated 
with an average effect size of about .75. In those cases where 
5 of the 6 adult learning method characteristics were used, the 
average effect size was almost 1.25. The latter indicates that 
there is value-added benefits of adult learning methods that 
included multiple characteristics. 

Moderators of the Effects 
of the Adult Learning Methods 

Whether the relationships between the adult learning 
method characteristics were moderated by study, learner, set- 
ting, or intervention variables was assessed by constituting 
contrasting moderator variable groups and examining the 
sizes of effects of the independent and dependent measures 
for each moderator group. The study variables that were ex- 
amined included research design and study sample size. One 
learner variable was examined: College students vs. non-col- 
lege students (practitioners and English language learners). 
The effect of setting was examined by comparing the use of 
the adult learning methods in college classrooms vs. work 
settings. One intervention variable was examined as a mod- 
erator: Length of the learning experience. 

Table 4 shows the result of the analyses. The relation- 
ships between the adult learning methods and the study out- 



NUMBER OF ADULT LEARNING METHOD CHARACTERISTICS 

Figure 3. Average effect sizes and 95% confidence interval 
for the relationship between the number of the different 
adult learning method characteristics included in a study 
and the learner outcomes. 

comes were all significantly related to learner benefits re- 
gardless of moderator variable. The strength of the relation- 
ships, however, were either similar or different depending on 
the moderator. The differences in the research designs, types 
of study, settings, and the learners constituting the focus of 
investigation did not influence the relationship between the 
independent and outcome variables. The relationship be- 
tween the independent and dependent variables did differ as 
a function of both study sample size and length of training or 
instruction. The adult learning methods were more effective 
when used with a relatively small number of learners where 
the learning experience occurred for 10 or more hours. 

Discussion 

The extent to which the characteristics of four different 
adult learning methods were associated with improved learner 
outcomes was the focus of the research synthesis presented 
in this paper. Findings described in How People Learn 
(Bransford et al., 2000; Donovan et al., 1999) were used to 
operationally define six adult learning method characteristics 
and to code the use or presence of each characteristic in the 
studies included in the research synthesis (see also Graham, 
1989). Results showed that all six characteristics were 
associated with more positive learner outcomes, and that 
adult learning method practices that more actively involved 
learners in using, processing, and evaluating their knowledge 
and skills as part of learning and mastering new information 
or practices were most effective. The optimal benefits of the 
adult learning methods were realized when the majority of 
the adult learning method characteristics were simultaneously 
used in an intervention, the intervention was implemented 
with a small number of learners, and the learning experience 
was implemented for more than 10 hours on different 
occasions. 

The findings reported in this paper, taken together, are 
consistent with the science of learning described by Brans- 
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Table 4 

Moderators of the Relationships Between the Adult Learning Method Characteristics and the Study Outcomes 




Number 

Mean Rffect 

95% Confidence 

Moderators 

Studies 

Effect Sizes 

(d) Size 

Interval 

Research Design 

Randomized Studies 

58 

175 

.56 

.41-. 71 

Comparative Group Studies 

21 

39 

.64 

.46-. 82 

Type of Study 

Published 

43 

81 

.67 

.41-. 93 

Unpublished 

36 

133 

.52 

.39-. 64 

Sample Size 

Small (9 to 34) 

21 

88 

.76 

.52-1.00 

Medium (35 to 75) 

30 

74 

.46 

.28-. 64 

Large (76 to 300+) 

25 

49 

.37 

.21-. 53 

Learners 

College Students 

46 

83 

.49 

.33-. 65 

Practitioners 

33 

131 

.63 

.45-. 81 

Setting 

Classroom 

62 

167 

.55 

.41-. 69 

Work 

15 

42 

.50 

.33-. 67 

Length of Training 

1 to 10 Hours 

24 

57 

.30 

.10-. 50 

11 to 40 Hours 

28 

101 

.73 

.52-. 95 

40+ Hours 

25 

49 

.66 

.49-. 82 


ford et al. (2000) and Donovan et al. (1999). The findings are 
also consistent with the basic tenets of most adult learning 
theories (e.g., Knowles et al., 1998; Merriam, 1987; Trot- 
ter, 2006; Yang, 2003). Results from the research synthesis 
extend previous theory and research by isolating what mat- 
ters most in terms of adult learning and the conditions under 
which the benefits of adult learning methods are likely to be 
optimally effective. According to Bransford et al. (2000), one 
difference between novice and expert learners is the ability to 
understand new knowledge or practice in the context of some 
conceptual or operational framework which leads to deeper 
understanding of the knowledge or practice. This is often not 
an explicit part of how training is implemented or instruc- 
tion is provided, yet the indicators of deep understanding 
examined in this research synthesis were found to be the par- 
ticular characteristics most strongly associated with learner 
outcomes (Table 3). Indeed, engaging learners in a process 
of self-assessment of their performance using some type of 
conceptual or operational framework proved to be a practice 
that resulted in the largest sizes of effects between the adult 
learning method characteristics and the learner outcomes. 

A finding from the research synthesis that is not explicit 
but deserves comment is the fact that the results contradict 
tenets of some theories of learning and some adult learning 
methods that emphasize learner discovery as a key element 
of the learning process (e.g., Denis, 2008; Hanft, Rush, & 
Shelden, 2004; Ozuah, 2005; Quay, 2003); namely learner 
self-discovery in the absence of instructor guidance or feed- 


back, a characteristic of individualistic constructivism (Smith 
& Ragan, 1999). According to Bransford et al. (2000), 

A common misconception [of] “constructionist” 
theories of knowing (that existing knowledge is 
used to build new knowledge) is that teachers should 
never tell [learners] anything directly, but, instead, 
should always allow them to construct knowledge 
for themselves. This perspective confuses a theory 
of pedagogy (teaching) with a theory of knowing .... 
There are times when [instructor guided learning] 
can work extremely well. (p. 11) 

The latter was found to be the case in this research synthesis 
where the use of practices by instructors that facilitated learn- 
er knowledge acquisition and mastery helped learners assess 
their learning experience or judge their experience against 
some set of standards or criteria. Both practices proved an 
important element of optimal learner outcomes. Learners 
are not likely to become experts without instructors engag- 
ing them in a process of evaluating their experiences in the 
context of some framework, model, or operationally defined 
performance standards or expectations (e.g., Henry, McTag- 
gert, & McMillan, 1992; Otis-Wilborn, Winn, & Ford, 2000). 
Learner discovery in the absence of guided instruction would 
therefore appear misguided. 

It is also worth noting which adult learning method 
practices were not effective or were only minimally effec- 
tive. Imagery and dramatic readings, alone or in combination, 
were generally ineffective for introducing new information 
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or practice to learners, and (passively watching) instructional 
videos was not the most effective approach for illustrating or 
demonstrating the application of new information or practice. 
The former (imagery and dramatic readings) are clearly not 
warranted as part of adult learning, whereas the latter (instruc- 
tional videos) are perhaps effective when used in combination 
with other methods of illustration (e.g., learner input). 

Implications for Practice 

This research synthesis was conducted with the aim of 
identifying those adult learning method characteristics that 
could be used to inform changes and improvements in inser- 
vice training opportunities. The findings clearly have direct im- 
plications for this purpose. The results also have implications 
for other kinds of instruction with both adults and children. 
The implications for inservice training include the following: 

• The six adult learning method characteristics constituting 
the focus of investigation provide guidance and structure 
for developing effective training and technical assistance 
programs and practices. The most effective training is 
likely to include learner experiences and opportunities 
in each of the three main components of adult learning 
(planning, application, and deep understanding-Table 1). 

• The more adult learning method characteristics that are 
incorporated into a training program or practice, the 
more likely the learning experiences will have optimal 
positive benefits (Figure 3). The six characteristics each 
provide a different vantage point for promoting learner 
acquisition, use, and evaluation of new knowledge, ma- 
terial, or practice. 

• The common element of adult learning methods that 
are most effective is active learner participation in the 
learning process. This is consistent with both the basic 
tenets of adult learning theory as well as research show- 
ing that active participation in learning new knowledge 
or practice has value-added benefits. 

• Training opportunities are likely to be most effective if 
they include multiple learning experiences, large doses 
of learner self-assessment of their experiences, and in- 
structor facilitated learner assessment of his or her learn- 
ing against some set of standards or criteria (Table 3). 
The more opportunities a learner has to acquire and use 
new knowledge or practice, the more frequently those 
opportunities occur, and the more the learner is engaged 
in reflection on those opportunities using some external 
set of standards, the greater the likelihood of optimal 
benefits. 

• To the extent possible, the training provided to learn- 
ers should include a small number of participants where 
the training is provided on multiple occasions (Table 4). 
The fewer the number of learners, the more likely the 
instructor can give the necessary attention to the largest 
majority of learners. The more occasions the training is 
provided, the more opportunities for processing, reflec- 
tion, and assessment of mastery. 

These, as a minimum, are indicated as the focus of training 
based on the results reported in this paper. 


The type of professional development (training, techni- 
cal assistance, etc.) suggested by the findings in this research 
synthesis would necessitate changes in both the organization 
of training and the ways in which learners are involved in 
training opportunities. On the one hand, the results indicate 
that professional development includes multiple opportunities 
to learn and master new knowledge, material, and practices, 
and that any one opportunity includes varied experiences to 
leam, practice, and process the target of training. On the other 
hand, the results indicate that learners be as actively involved 
as possible in all aspects of the training experiences. 

Further implications for professional development have 
to do with the role teachers and trainers play in promoting 
learning. Professional development as currently practiced falls 
along a continuum from one-time didactic workshops to dis- 
covery and experiential learning. Findings from our research 
synthesis “point to” a middle ground where professional de- 
velopment personal structure learning opportunities where 
learners are provided guidance, feedback, support, etc. to 
develop deeper understanding of the focus of learning. Find- 
ings reported in How People Learn (Bransford et ah, 2000; 
Donovan et ah, 1999) as well as elsewhere (e.g., Kirschner, 
Sweller, & Clark, 2006) indicate, as was found in our research 
synthesis, that guiding but not directing learning can promote 
and facilitate mastery of new knowledge or practice. 
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Appendix A 


Number of Study Participants and Learner Characteristics and Outcomes 





Adult 





Sample Size 3 

Learning 

Learner 



Study 

E 

C 

Method' 

Description 

Setting 

Learner Outcome 

Anderson & Render (1987) Study 1 

56 

54 

AL 

Undergraduate 

College classroom 

Rare vocabulary 

Anderson & Render (1987) Study 2 

23 

54 b 


students 



Anderson & Render (1987) Study 3 

26 

54 b 





Anderson & Render (1987) Study 4 

20 

54 b 





Baker (1983) 

4 

7 

CH 

Teachers 

Junior high school 

Teaching models 

Bartley (1997) 

24 

26 

JIT 

Undergraduate 

College classroom 

Microcomputers 





students 



Bass (1985) Study 1 

19 

20 

AL 

Undergraduate 

College classroom 

Rare vocabulary 

Bass (1985) Study 2 

19 

20 b 


students 



Benedict & Anderton (2004) 

56 

67 

JIT 

Undergraduate 

College classroom 

Statistics 





students 



Bing- You et al. (1998) 

36 

41 

CH 

Medical 

Hospital 

Effective feedback 





students 


from residents/doctors 

Bodine (1978) 

75 

62 

GD 

Undergraduate 

College classroom 

Biochemistry 





students 



Bowman & McCormick (2000) 

16 

16 

CH 

Pre-service 

Elementary school 

Instructional strategies 





teachers 


and pedagogical models 

Bradner (1996) 

36 

26 

AL 

Customer 

New employee 

Telephone 





service 

orientation workshop 

customer service 

Bullard (1986) 

12 

11 

GD 

Undergraduate 

College classroom 

Electrical 





students 


engineering 

Cain et al. (2007) 

24 

24 

CH 

Teachers 

Preschool 

Joint attention 







techniques 

Campbell (1984) 

18 

18 

GD 

Undergraduate 

College classroom 

French 





students 



Campbell (1986) 

69 

61 

GD 

Undergraduate 

College classroom 

Spanish 





students 



Caux (1995) 

12 

11 

AL 

Undergraduate 

College classroom 

French 





students 



Clerici-Arias et al. (2003) 

42 

42 

JIT 

Undergraduate 

College classroom 

Economics 





students 



Coscarelli & White (1982) 

321 

228 

GD 

Undergraduate 

College classroom 

Production/operation 





students 


management 

Craven (1990) Study 1 

9 

4 

CH 

Teachers 

Junior high school 

Effective instructional 

Craven (1990) Study 2 

5 

4 b 




practices 

Dipamo & Job (1990, 1991) 

12 

12 

AL 

Undergraduate 

College classroom 

Rare vocabulary 





students 



Du Babcock (1986, 1988) Study 1 

20 

320 

AL 

ESL students 

Community college 

English as a Second 

Du Babcock (1986, 1988) Study 2 

14 

320 b 



classroom 

Language 

Earl (1993) 

33 

31 

AL 

Managers 

Managerial workshop 

Management skills 

Eastman (1993) 

41 

40 

AL 

Accountants 

Industry training class 

Tax reporting 

Edwards (1993) 

16 

19 

CH 

Teachers 

Preschool, elementary. 

Cognitive coaching 






junior high or 
secondary school 


Edwards et al. (1998) 

240 

100 

CH 

Teachers 

Elementary, junior 
high or secondary 

Cognitive coaching 






school 


Edwards & Newton (1995) 

44 

88 

CH 

Teachers 

Public school 

Cognitive coaching 
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Appendix A, continued 


Study 

Sample Size 3 
E C 

Adult 

Learning 

Method 3 

Learner 

Description 

Setting 

Learner Outcome 

Garcia (1984 

40 

40 

AL 

ESL students 

Community 

classroom 

English as a Second 
Language 

Gattellari et al. (2005) 

135 

140 

CH 

Physicians 

Medical office 

PSA screening 
practices 

Gavrin et al. (2004) 

NR 

NR 

JIT 

Undergraduate 

students 

College classroom 

Physics 

Goker (2006) 

16 

16 

CH 

Teachers 

Not reported 

Instructional practice 

Goldberg & Shuman (1984a) 

53 

80 

GD 

Undergraduate 

students 

College classroom 

Physical science 

Goldberg & Shuman (1984b) 

57 

41 

GD 

Undergraduate 

students 

College classroom 

Physical science 

Goldsmith et al. (2000) 

98 

97 

JIT 

Patients 

Hospital 

Pain management 

Hancock et al. (1981, 1983) 

321 

228 

GD 

Undergraduate 

students 

College classroom 

Production 

management 

Hepner (1989) 

21 

20 

GD 

Undergraduate 

students 

College classroom 

Nursing 

Hoggard (1980) 

30 

NR 

GD 

Undergraduate 

students 

College classroom 

Chemistry 

Hosack-Curlin (1988) 

12 

12 

CH 

Teachers 

Elementary school 

Writing process 
instruction 

Howard (2004) 

60 

112 

JIT 

Undergraduate 

students 

College classroom 

Sociology 

Hursh et al. (1980) 

26 

24 

GD 

Undergraduate 

students 

College classroom 

Engineering 

Landers (1975) 

27 

15 

GD 

Undergraduate 

students 

College classroom 

Engineering 

McGinty (1988) 

16 

19 

AL 

Undergraduate 

students 

College classroom 

Administrative policy 
business 

Meyer (1997) Study 1 

20 

11 

AL 

Undergraduate 

students 

College classroom 

Criminal justice - 
American courts 
system 

Meyer (1997) Study 2 

14 

6 

AL 

Undergraduate 

students 

College classroom 

Research methods 
and statistics 

Miller et al. (2004) Study 1 
Miller et al. (2004) Study 2 
Miller et al. (2004) Study 3 

33 

34 
26 

23 

23 b 

23 b 

CH 

Counselors 

Human services and 
health office 

Motivational 
interviewing technique 

Moreno-Montalvo (1987) 

31 

28 

AL 

ESL students 

College classroom 

English as a Second 
Language 

Newsome & Tillman (1990) 

25 

25 

GD 

Undergraduate 

students 

College classroom 

Nursing 

O’Connor & Korr (1996) 

9 

9 

CH 

Teachers 

Elementary school 

Teacher ability to 
address students’ need 

Peterson (1996) 

24 

18 

AL 

Managers 

Cost management 
workshop 

Cost management 

Pierce & Miller (1994) 

14 

15 

CH 

Pre-service teachers 

Special education 
classroom 

Teaching practices 

Portes et al. (1992) 

81 

74 

AL 

Undergraduate 

students 

College classroom 

Education psychology 
& mathematics 

Preziosi (1995) Study 1 
Preziosi (1995) Study 2 

33 

35 

37 

34 

AL 

Graduate 

students 

College classroom 

Management 

productivity/ 

improvement 
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Appendix A, continued 



Sample Size 3 

Adult 
- Learning 
Method' 

Learner 

Description 



Study 

E 

C 

Setting 

Learner Outcome 

Prichard (1990) 

28 

21 

AL 

Undergraduate 

students 

College classroom 

Math 

Pugach & Johnson (1995) 

95 

96 

CH 

Teachers 

Not reported 

Classroom 
problem solving 

Robinett (1976) Study 1 
Robinett (1976) Study 2 

66 

14 

18 

18 b 

AL 

Undergraduate 

students 

College classroom 

Spanish 

Schiffler (1986) Study 1 

18 

18 

AL 

Undergraduate 

students 

College classroom 

French 

Schiffler (1986) Study 2 

11 

11 

AL 

Undergraduate 

students 

College classroom 

French 

Schuster (1976) 

19 

32 

AL 

Undergraduate 

students 

College classroom 

Spanish 

Sears (1973) 

16 

19 

GD 

Undergraduate 

students 

College classroom 

Thermodynamic s 
kinetics 

Shaw (1980) 

21 

NR 

GD 

Undergraduate 

students 

College classroom 

Not reported 

Showers (1982) 

9 

8 

CH 

Teachers 

Junior high school 

Teaching models 

Showers (1984) 

10 

5 

CH 

Teachers 

Junior high school 

Teaching models 

Simkins & Maier (2004) 

18 

19 

JIT 

Undergraduate 

students 

College classroom 

Macroeconomics 

Slunt & Giancarlo(2004) 

66 

185 

JIT 

Undergraduate 

students 

College classroom 

Organic chemistry 

Snyder (1980) 

33 

20 

GD 

Undergraduate 

students 

College classroom 

Medical surgical 
procedures 

Sparks (1986) 

6 

6 

CH 

Teachers 

Junior high school 

Time use 

Stahl etal. (1991) Study 1 
Stahl et al. (1991) Study 2 

20 

20 

20 

20 b 

AL 

Undergraduate 

students 

College classroom 

Rare vocabulary 

Stein etal. (1982) 

18 

24 

AL 

Graduate 

students 

College classroom 

Rare vocabulary 

Streufert (1985) 

13 

16 

CH 

Teachers 

Elementary school 

Reading program 
refresher 

Wynn (1987) 

11 

11 

CH 

Pre-service teachers 

Elementary school 

Teaching methods 

Zeiss (1984) 

7 

7 

AL 

ESL students 

College classroom 

English as a Second 
Language 


a E = Experimental group and C = Control or comparison group. 

b Indicates that the same control or comparison group was used as the contrast for the different experimental groups. 
C AL = Accelerated learning, CH = Coaching, GD = Guided design, and JIT = Just-in-time training. 
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Appendix B 


Adult Learning Method Practice Characteristics 



Length of 

Planning 

Application 

Understanding 

Study 

Training 

(Hrs) 

Introduce 

Illustrate 

Practice Evaluate 

Reflection 

Mastery 

Anderson & Render 
(1987) 

< 1 

Dramatic 

reading 

NR 11 

NR NR 

NR 

NR 

Baker (1983) 

26 

Lecture 

Role play 

Real life Discussion of 

strengths and 
weaknesses 

Performance 

improvement 

NR 

Bartley (1997) 

48 

Quizzes 
to probe 
knowledge 
prior to 
lecture 

NR 

NR NR 

NR 

NR 

Bass (1985) 

11 

Imagery and 
dramatic 
reading 

NR 

Learner Learners check 

participation in their own quiz 
word games or answers 

creative writing 

NR 

NR 

Benedict & Anderton 
(2004) 

48 

Quizzes 
to probe 
knowledge 
prior to 
lecture 

Answers 
from quizzes 
incorporated 
into class 
lectures 

NR NR 

NR 

NR 

Bing- You et al. (1998) 

2 

Lecture 

Instructional 

video 

Role play and NR 

real life 

NR 

Skills based 
assessment 

Bodine (1978) 

8 

Lecture 

NR 

Realistic Review solution 

problem-solving that group 

situations or individual 

provided and 
make corrections 
to proposed 
solution 

Group 
reflection on 
feedback 

Group assesses 
feedback 
received from 
instructor 

Bowman & McCormick 
(2000) 

21 

Lecture 

Role play 

Role play NR 

and real life 

Performance 

improvement 

Skills based 
assessment 

Bradner (1996) 

64 

Dramatic 
reading and 
peripherals 

Real life 

Skits, plays NR 

and role play 

NR 

NR 

Bullard (1986) 

16 

Learner self- 
instruction 

NR 

Realistic Review solution 

problem-solving that group 

situations or individual 

provided and 
make corrections 
to proposed 
solution 

Group 
reflection on 
feedback 

Group assesses 
feedback 
received from 
instructor 

Campbell (1984) 

9 

Learner self- 
instruction 

NR 

Realistic Review solution 

problem-solving that group 

situations or individual 

provided and 
make corrections 
to proposed 
solution 

Group 
reflection on 
feedback 

Group assesses 
feedback 
received from 
instructor 
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Appendix B, continued 


Length of Planning Application Understanding 

Training 

Study (Hrs) Introduce Illustrate Practice Evaluate Reflection Mastery 


Campbell (1986) 

48 

Learner self- 
instruction 

NR 

Caux (1995) 

80 

Peripherals 

NR 

Clerici-Arias et al. 
(2003) 

50 

Activity to 
complete and 
submit prior 
to lecture 

Answers 
from quizzes 
incorporated 
into class 
lectures 

Coscarelli & White 
(1982) 

48 

Learner self- 
instruction 

Simulation 


Craven (1990) Study 1 

16 

Lecture 

Instructional 

video 

Craven (1990) Study 2 

11 

Lecture 

Instructional 

video 

Dipamo & lob (1990, 
1991) 

< 1 

Imagery 

Real life 

Du Babcock (1986, 
1988) 

60 

Dramatic 
reading and 
peripherals 

Role play 

Earl (1993) 

3 

Imagery 

NR 

Eastman (1993) 

8 

Imagery 

NR 

Edwards (1993) 

9 

NR 

NR 

Edwards et al. (1998) 

150 

NR 

NR 

Edwards & Newton 
(1995) 

NR 

NR 

NR 

Garcia (1984) 

16 

Dramatic 

reading 

Role play 

Gattellari et al. (2005) 

3 

Lecture 

NR 


Realistic 

Review solution 

Group 

Group assesses 

problem- 

that group 

reflection on 

feedback 

solving 

or individual 

feedback 

received from 

situations 

provided and 
make corrections 
to proposed 
solution 


instructor 

NR 

NR 

NR 

NR 

Learner 
participation 
in guided 

Discussions of 
strengths and 
weaknesses 

NR 

NR 

discussion or 
activities based 
on responses 
to pre-class 
assignments 

Realistic 

Review solution 

Group 

Group assesses 

problem- 

that group 

reflection on 

feedback 

solving 

or individual 

feedback 

received from 

situations 

provided and 
make corrections 
to proposed 
solution 


instructor 

Real Life 

Discussions of 
strengths and 
weaknesses 

NR 

NR 

Real life 

NR 

NR 

NR 

Learner 

NR 

NR 

NR 

participation in 
word games or 
creative writing 

Skits, plays 
and role play 

Instructor 
provides mild 
evaluative 
statement of 
mistakes 

NR 

NR 

NR 

NR 

NR 

NR 

NR 

NR 

NR 

NR 

Real life 

NR 

Performance 

improvement 

Skills based 
assessment 

Real life 

NR 

Performance 

improvement 

Skills based 
assessment 

Real life 

NR 

Performance 

improvement 

Skills based 
assessment 

Skits, plays 

NR 

NR 

NR 

and role play 

Real life 

NR 

NR 

Skills based 
assessment 
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Length of 
Training 
(Hrs) 

Planning 

Application 

Understanding 

Study 

Introduce 

Illustrate 

Practice 

Evaluate 

Reflection 

Mastery 

Gavrin et al (2004) 

48 

Warm up 
exercise 
completed 
less than 12 
hours prior 
to class 

Answers 
from quizzes 
incorporated 
into class 
lectures 

Learner 
participation 
in guided 
discussion or 
activities based 
on responses to 
pre-class 
assignments 

NR 

NR 

NR 

Goker (2006) 

21 

Lecture 

Role play 

Real life 

Discussion of 
strengths and 
weakness 

NR 

NR 

Goldberg & Shuman 
(1984a) 

96 

Lecture 

Simulation 

Realistic 

problem-solving 

situations 

Review solution 
that group 
or individual 
provided and 
make corrections 
to proposed 
solution 

Group 
reflection on 
feedback 

Group assesses 
feedback 
received from 
instructor 

Goldberg & Shuman 
(1984b) 

13 

Lecture 

Simulation 

Realistic 

problem-solving 

situations 

Review solution 
that group 
or individual 
provided and 
make corrections 
to proposed 
solution 

Group 
reflection on 
feedback 

Group assesses 
feedback 
received from 
instructor 

Goldsmith et al. (2000) 

1 

Information 
given to 
patient by 
nurse 

NR 

Access 

information on 
website 

NR 

NR 

NR 

Hancock (1981, 1983) 

48 

Learner self- 
instruction 

NR 

Realistic 

problem-solving 

situations 

Review solution 
that group or 
individual 
provided 

Group 
reflection on 
feedback 

Group assesses 
feedback 
received from 
instructor 

Hepner (1989) 

12 

Lecture 

NR 

Realistic 

problem-solving 

situations 

Review solution 
that group or 
individual 
provided 

Group 
reflection on 
feedback 

Group assesses 
feedback 
received from 
instructor 

Hoggard (1980) 

48 

Lecture 

NR 

Realistic 

problem-solving 

situations 

Review solution 
that group 
or individual 
provided and 
make corrections 
to proposed 
solution 

Group 
reflection on 
feedback 

Group assesses 
feedback 
received from 
instructor 

Hosack-Curlin (1988) 

38 

Lecture 

Role play 
and real life 

Role play 
and real life 

Discussion of 
strengths and 
weakness 

Journal 

writing 

NR 

Howard (2004) 

48 

Warm up 
exercise 
completed 
less than 12 
hours prior 
to class 

Answers 
from quizzes 
incorporated 
into class 
lectures 

NR 

NR 

NR 

NR 
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Length of 
Training 
(Hrs) 

Planning 

Application 

Understanding 

Study 

Introduce 

Illustrate 

Practice 

Evaluate 

Reflection 

Mastery 

Hursh et al. (1980) 

48 

Learner self- 
instruction 

NR 

Realistic 
problem solving 
situations 

Review solution 
that group 
or individual 
provided and 
make corrections 
to proposed 
solution 

Group 
reflection on 
feedback 

Group assesses 
feedback 
received from 
instructor 

Landers (1975) 

30 

Lecture 

NR 

Realistic 

problem-solving 

situations 

Review solution 
that group 
or individual 
provided 

Group 
reflection on 
feedback 

Group assesses 
feedback 
received from 
instructor 

McGinty (1988) 

36 

Dramatic 

reading 

NR 

Skits, plays 
and role play 

NR 

Journal 

writing 

NR 

Meyer (1997) Study 1 

2 

Imagery and 
peripherals 

Real life 

Learner 
participation in 
word games or 
creative writing 

Instructor 
provides mild 
evaluative 
statement of 
mistakes 

NR 

NR 

Meyer (1997) Study 2 

2 

Imagery and 
peripherals 

Real life 

Learner 
participation in 
word games or 
creative writing 

Instructor 
provides mild 
evaluative 
statement of 
mistakes 

NR 

NR 

Miller et al. (2004) 
Study 1 

13 

Lecture 

Role play 

Role play 

NR 

NR 

Standards based 
assessment 

Miller et al. (2004) 
Study 2 

18 

Lecture 

Role play 

Role play 
and real life 

NR 

Behavior 

suggestions 

Skills based 
assessment 

Miller et al. (2004) 
Study 3 

19 

Lecture 

Role play 

Role play 
and real life 

NR 

Behavior 

suggestions 

Standards based 
assessment 

Moreno-Montalvo 

(1987) 

39 

Imagery, 
dramatic 
reading and 
peripherals 

NR 

Skits, plays 
and role play 

NR 

NR 

NR 

Newsome & Tillman 
(1990) 

9 

Learner self- 
instruction 

NR 

Realistic 

problem-solving 

situations 

Review solution 
that group 
or individual 
provided 

Group 
reflection on 
feedback 

Group assesses 
feedback 
received from 
instructor 

O’Connor & Korr 
(1996) 

10 

Lecture 

NR 

Real life 

Discussion of 
strengths and 
weaknesses 

Behavior 

suggestions 

NR 

Peterson (1996) 

NR 

Imagery, 
dramatic 
reading and 
peripherals 

NR 

Skits, plays 
and role play 

NR 

NR 

NR 

Pierce & Miller (1994) 

65 

Lecture 

NR 

Real life 

NR 

Performance 

improvement 

Skills based 
assessment 

Portes et al. (1992) 

3 

Imagery 

NR 

NR 

NR 

NR 

NR 

Preziosi (1995) 

20 

Peripherals 

NR 

NR 

NR 

NR 

NR 

Prichard (1990) 

68 

Dramatic 

reading 

NR 

NR 

NR 

NR 

NR 


Practical Evaluation Reports 


22 


Volume Two, Number One, March 2009 


Appendix B, continued 



Length of 
Training 
(Hrs) 

Planning 

Application 

Understanding 

Study 

Introduce 

Illustrate 

Practice 

Evaluate 

Reflection 

Mastery 

Pugach & Johnson 
(1995) 

7 

Lecture 

Instructional 

video 

Real life 

NR 

Behavior 

suggestions 

Skills based 
assessment 

Robinett (1975) 

25 

Dramatic 

reading 

NR 

NR 

NR 

NR 

NR 

Schiffler (1986) Study 1 

56 

Dramatic 

reading 

Role play 

Role play 

NR 

NR 

NR 

Schiffler (1986) Study 2 

56 

Dramatic 

reading 

Role play 

Role play 

NR 

NR 

NR 

Schuster (1976) 

32 

Dramatic 

reading 

Role play 

Role play 

NR 

NR 

NR 

Sears (1973) 

96 

Learner self- 
instruction 

NR 

Realistic 

problem-solving 

situations 

Review solution 
that group 
or individual 
provided 

Group 
reflection on 
feedback 

Group assesses 
feedback 
received from 
instructor 

Shaw (1980) 

48 

Learner self- 
instruction 

Simulation 

Realistic 

problem-solving 

situations 

Review solution 
that group 
or individual 
provided and 
make corrections 
to proposed 
solution 

Group 
reflection on 
feedback 

Group assesses 
feedback 
received from 
instructor 

Showers (1982) 

26 

Lecture 

Role play 

Real life 

Discussion of 
strengths and 
weaknesses 

Performance 

improvement 

NR 

Showers (1984) 

25 

Lecture 

Role play 

Real life 

Discussion of 
strengths and 
weaknesses 

Performance 

improvement 

NR 

Simkins & Maier (2004) 

48 

Activity to 
complete and 
submit prior 
to lecture 

Answers 
from quizzes 
incorporated 
into class 
lectures 

NR 

NR 

NR 

NR 

Slunt & Giancarlo 
(2004) 

48 

Warm up 
exercise 
completed 
less than 12 
hours prior to 
class 

Answers 
from quizzes 
incorporated 
into class 
lectures 

Learner 
participation 
in guided 
discussion or 
activities based 
on responses to 
pre-class 
assignment 

NR 

NR 

NR 

Snyder (1980) 

48 

Learner self- 
instruction 

NR 

Realistic 
problem solving 
situations 

Review solution 
that group 
or individual 
provided and 
make corrections 
to proposed 
solution 

Group reflection Group assesses 
on feedback feedback 

received from 
instructor 

Sparks (1986) 

12 

Lecture 

Role play 

Real life 

Discussion of 
strengths and 
weaknesses 

Behavior 

suggestions 

NR 

Stahl et al. (1991) 
Study 1 

18 

Imagery 

NR 

NR 

NR 

NR 

NR 
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Length of 
Training 
(Hrs) 

Planning 

Application 

Understanding 

Study 

Introduce 

Illustrate 

Practice 

Evaluate 

Reflection 

Mastery 

Stahl etal. (1991) 
Study 2 

18 

Dramatic 

reading 

NR 

NR 

Learners check 
their own quiz 
answers 

NR 

NR 

Stein etal. (1982) 

< 1 

Dramatic 

reading 

NR 

NR 

NR 

NR 

NR 

Streufert (1984) 

7 

Lecture 

Real life 

Real life 

NR 

Performance 

improvement 

Skills-based 

assessment 

Wynn (1986) 

30 

Lecture 

Instructional 

video 

Real life 

Discussion of 
strengths and 
weaknesses 

Journal writing 

NR 

Zeiss (1984) 

13 

Dramatic 

reading 

NR 

NR 

NR 

NR 

NR 


“Not reported or described as a characteristic of the adult learning method. 
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Major Findings from the Studies of the Adult Learning Methods 



Adult 







Learning 

Type of 

Outcome 

Type of 


Cohen’s d 

Study 

Method 3 

Study b 

Construct 

Measure 

Outcome Measures 

Effect Size 

Anderson & Render (1987) 
Study 1 

AL 

RCT 

Knowledge 

Word to definition 

Language posttest exam 

-1.14 

Anderson & Render (1987) 
Study 2 

AL 

RCT 

Knowledge 

Word to definition 

Language posttest exam 

-1.29 

Anderson & Render (1987) 
Study 3 

AL 

RCT 

Knowledge 

Word to definition 

Language posttest exam 

-1.04 

Anderson & Render (1987) 
Study 4 

AL 

RCT 

Knowledge 

Word to definition 

Language posttest exam 

-1.07 

Baker (1983) 

CH 

RCT 

Skills 

Observational 

Transfer of Training: Skills 

1.77 





rating scale 

Transfer of Training: Appropriateness 

1.64 






Transfer of Training: Comfort/ 
familiarity 

1.49 






Transfer of Training: Practice/ 
frequency of use 

1.18 




Skills 

Teacher Innovator 

Structuring skills 

.86 





System (Weil, 
Gullion, & Cole, 
1971) 

Information processing skills 

1.05 





Feedback skills 

.41 





Factual Information Processing 

.41 






Conceptual Information Processing 

1.07 






Theoretical Information Processing 

.78 

Bartley (1997) 

JIT 

RCT 

Knowledge 

Course content 

Teacher constructed knowledge test 

1.15 

Bass (1985) Study 1 

AL 

RCT 

Attitude 

Listed Thought 
Procedure (Bass, 
1985) c 

LPT attitude scores 

.24 




Knowledge 

Word to definition 

Language vocabulary exam 

.45 

Bass (1985) Study 2 

AL 

RCT 

Attitude 

Listed Thought 
Procedure (Bass, 
1985) 

LPT attitude scores 

.15 




Knowledge 

Word to definition 

Language vocabulary exam 

.44 

Benedict & Anderton 
(2004) 

JIT 

RCT 

Knowledge 

Course content 

Final exam 

.38 

Bing- You et al. (1998) 

CH 

RCT 

Self-efficacy 

Survey 

How I am progressing 

.38 






Enough information to improve my 
performance 

.11 






How I compare to my peers 

-.09 






Enough information to know I am a 
competent student 

.54 






Effective in soliciting the feedback I 
need 

-.13 






How to develop personal learning goals 

.24 

Bodine (1978) 

GD 

RCT 

Attitude 

Survey 

Study attitudes toward college courses 

.39 




Knowledge 

Course content 

Content knowledge exam 

-.12 

Bowman & McCormick 

CH 

RCT 

Skills 

Clarity 

Frequency of clarity skills 

1.78 

(2000) 




Observation 

Instrument 

Quality of use of clarity skills 

2.57 





(Metcalf, 1989) 

Overall demonstration of clarity skills 

2.36 




Knowledge 

Observation 

Knowledge of education theory 

5.84 




Attitude 

Rating Scale 

Technical feedback 

.79 
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Adult 

Learning 

Type of 

Outcome 

Type of 


Cohen’s d 

Study 

Method 3 

Study b 

Construct 

Measure 

Outcome Measures 

Effect Size 

Bowman & McCormick 

CH 

RCT 

Attitude 

Rating Scale 

Analysis of application 

1.12 

(2000), continued 





Adaptation to students 

.86 






Personal facilitation 

.85 

Bradner (1996) 

AL 

RCT 

Knowledge 

Course content 

Test of job requirements 

.77 





Observation rating 

Customer service test 

.44 




Skills 

Course content 

Computer usage exam 

.47 

Bullard (1986) 

GD 

RCT 

Knowledge 

Course content 

Content laboratory knowledge exam 

1.16 

Cain et al (2007) 

CH 

RCT 

Skills 

Joint Attention 

Joint attention engagement bids 

1.49 





Observation 
Coding System 
(Cain, Rudd, & 
Saxon, 2007) 

Verbal events 

-.82 





Focus-Follow-Talk™ statements 

-1.50 





Joint attention support statements 

-.92 





Questions 

.19 






Teacher-directed statements 

1.90 

Campbell (1984) 

GD 

NRG 

Knowledge 

Course content 

Content knowledge exam 

.39 

Campbell (1986) 

GD 

RCT 

Knowledge 

Modern Language 
Cooperation 
(Campbell, 1986) 

Reading test 

.13 





Foreign Language 
Test (Campbell, 
1986) 

Writing test 

.51 





Course content 

Content knowledge exam 

.08 

Caux (1995) 

AL 

RCT 

Knowledge 

Oral Proficiency 
Interview 
(Byrnes, Buck, & 
Thompson, 1989) 

Oral language interview 

.93 

Clerici-Arias et al. (2003) 

JIT 

RCT 

Knowledge 

Course content 

Exam score comparisons 

.22 

Coscarelli & White (1982) 

GD 

NRG 

Knowledge 

Course content 

Content knowledge exam 

.53 

Craven (1990) Study 1 

CH 

RCT 

Skills 

COKER 

Students initiate verbal interaction 

.60 





(Modified) (Coker 
& Coker, 1982a, 
1982b) 

Teacher amplifies and discusses 
student response 

1.43 





Students are involved 

1.77 






Prepares and/or uses various methods 
and techniques to present subject 
matter and encourages student 
participation 

-.65 






Promotes positive self-image in 
students 

2.06 






Is consistent and empathetic in the 
treatment of students 

.34 






Practices good human relations 

-.58 






Exhibits overall positive approach 

-.03 






Stimulates group discussion and 
individual participation 

.07 






Nurtures creativity and discovery 

.90 






Helps learners develop positive attitude 
toward self, encourages confidence and 
self-respect 

.97 






Seeks, accepts and uses student ideas 
as part of teaching procedures 

1.77 
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Adult 






Learning 

Type of Outcome 

Type of 


Cohen’s d 

Study 

Method 3 

Study b Construct 

Measure 

Outcome Measures 

Effect Size 

Craven (1990) Study 1, 

CH 

RCT Skills 

COKER 

Motivates students to ask questions 

-.13 

continued 



(Modified) (Coker 
& Coker, 1982a, 
1982b) 

Uses questions that lead students to 
analyze, synthesize and think critically 

.09 




Accepts varied student viewpoints and/ 
or asks students to extend or elaborate 
answers or ideas 

.68 







Demonstrates proper listening skills 

.84 





Provides feedback to learners on their 
cognitive performance 

1.60 





Uses positive reinforcement patterns 
with students 

.58 

Craven (1990) Study 2 

CH 

RCT Skills 

COKER 

Students initiate verbal interaction 

.05 




(Modified) (Coker 
& Coker, 1982a, 
1982b) 

Teacher amplifies and discusses 
student response 

-.20 




Students are involved 

-.13 





Prepares and/or uses various methods 
and techniques to present subject 
matter and encourages student 
participation 

-.26 





Promotes positive self-image in students 

-.71 





Is consistent and empathetic in the 
treatment of students 

-1.15 





Practices good human relations 

-1.22 





Exhibits overall positive approach 

-.18 





Stimulates group discussion and 
individual participation 

-.58 





Nurtures creativity and discovery 

-.71 





Helps learners develop positive attitude 
toward self, encourages confidence and 
self-respect 

2.09 





Seeks, accepts and uses student ideas 
as part of teaching procedures 

-.50 





Motivates students to ask questions 

-.48 





Uses questions that lead students to 
analyze, synthesize and think critically 

-.48 





Accepts varied student viewpoints and 
/ or asks students to extend or elaborate 
answers or ideas 

-.37 





Demonstrates proper listening skills 

.63 





Provides feedback to learners on their 
cognitive performance 

.10 





Uses positive reinforcement patterns 
with students 

.58 

Dipamo& Job (1990, 1991) 

AL 

RCT Knowledge 

Paired associate 

Word acquisition and retention test 

-.64 




nouns test 



Du Babcock (1986, 1988) 

AL 

NRG Knowledge 

English Language 

Posttest language scroes 

.32 

Study 1 



Test (Best & Ilyin, 
1976) 



Du Babcock (1986, 1988) 

AL 

NRG Knowledge 

English Language 

Posttest language scores 

1.42 

Study 2 



Test (Best & Ilyin, 
1976) 
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Adult 

Learning 

Type of 

Outcome 

Type of 


Cohen’s d 

Study 

Method 3 

Study b 

Construct 

Measure 

Outcome Measures 

Effect Size 

Earl (1993) 

AL 

NRG 

Self-efficacy 

Rating scale 

Level of confidence score gains 

.35 






Confidence level after 1 month 

.40 




Attitude 

Rating scale 

Positive evaluations of the workshop 

.89 




Knowledge 

Course content 

Knowledge score test gains 

.16 





multiple-choice 

test 

Knowledge score retention after 1 
month 

-.10 




Skills 

Case study 

Accuracy of interpretation of case 
study posttest exam scores 

.93 

Eastman (1993) 

AL 

RCT 

Knowledge 

Course content 

Posttest scores (multiple-choice items) 

.15 





Hypothetical 

situation 

(provided missing 
information) 

Transfer task scores 

.29 




Attitude 

Rating scale 

Evaluation of course content 

.58 






Overall attitude toward class 

.48 






Overall rating of class 

.52 

Edwards (1993) 

CH 

NRG 

Knowledge 

Hunt Paragraph 
Completion 
Method (Hunt, 
Butler, Noy, & 
Rosser, 1978) 

Conceptual growth 

-.34 





Teacher Reflective 

Thought 

(Simmons, 

Sparks, Starko, 
Pasch, & Colton, 
1989) 

Reflective pedagogical thinking 

-.53 

Edwards et al. (1998) 

CH 

NRG 

Self-efficacy 

Teacher Efficacy 

Teacher efficacy 

1.26 





Scale (Soodak & 
Podell, 1996) 

Personal teaching efficacy 

.09 





Outcome efficacy 

1.29 

Edwards & Newton (1995) 

CH 

NRG 

Self-efficacy 

Teacher Efficacy 

Teacher efficacy 

.70 





Scale (Gibson & 
Dembo, 1984) 

Personal teaching efficacy 

.54 





Outcome efficacy 

.61 

Garcia (1984) 

AL 

NRG 

Knowledge 

Bilingual Syntax 
Measure II 
(Garcia, 1984) 

Progress in English learning 

.70 

Gattellari et al. (2005) 

CH 

RCT 

Knowledge 

Survey 

Questionnaire on evidence base for 
PSA screening 

1.33 






Questionnaire on steps to help patient 
make informed decisions 

1.04 




Self-efficacy 

Survey 

Rating of confidence in promoting 
informed decision making 

1.20 





Provider 

Level of confidence in conflict 

.42 





Decision Process 
Assessment 
Instrument 
(Dolan, 1999) 

situations 


Gavrin et al (2004) 

JIT 

NRG 

Attitude 

Attrition 

Student attrition rate 

1.76 

Goker (2006) 

CH 

RCT 

Skills 

Clarity 
Observation 
Instrument 
(Metcalf, 1989) 

Clarity of instructional skills 

5.84 
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Study 

Adult 

Learning 

Method 3 

Type of 
Study b 

Outcome 

Construct 

Type of 
Measure 

Outcome Measures 

Cohen’s d 
Effect Size 

Goker (2006), continued 

CH 

RCT 

Attitude 

Rating scale 

Satisfaction with coaching 

2.58 

Goldberg & Shuman 
(1984a) 

GD 

RCT 

Self-efficacy 

Survey 

Perceptions of decision making ability 

.42 

Goldberg & Shuman 
(1984b) 

GD 

RCT 

Knowledge 

Rating Scale 

Knowledge of causes and solutions to 
energy problem 

.05 




Self-efficacy 

Rating Scale 

Confidence in the ability to understand 
information about energy problems 

-.04 

Goldsmith et al. (2000) 

JIT 

RCT 

Skills 

Rating Scale 

Level of pain 

.37 

Hancock (1981, 1983) 

GD 

RCT 

Knowledge 

Course content 

Content knowledge exam 

.56 

Hepner (1989) 

GD 

RCT 

Knowledge 

Course content 

Content recall exam 

-.05 




Skills 

Course content 

Nursing plans exam 

2.34 






Application test exam 

.85 

Hoggard (1980) 

GD 

RCT 

Attitude 

Frequency count 

Dropout rate 

1.42 

Hosack-Curlin (1988) 

CH 

RCT 

Skills 

Observational 
rating scale 

Teacher implementation of writing 
skills process 

.37 





Levels of 
Use (Loucks, 
Newlove, & Hall, 
1975) 

Interview/writing process 

1.11 

Howard (2004) 

JIT 

NRG 

Knowledge 

Frequency count 

Participation in reading of required 
material 

1.10 

Hursh et al. (1980) 

GD 

RCT 

Knowledge 

Course content 

Course performance exam 

2.98 




Attitude 

Survey 

Student attitudes towards course 

.74 






experience 







Student attitudes towards course 

.35 






process 


Landers (1975) 

GD 

RCT 

Knowledge 

Course content 

Content knowledge exam 

.35 




Skills 

Application 

Performance projects 

-.01 

McGinty (1988) 

AL 

RCT 

Knowledge 

Cumulative class 
points 

Participation, mid-term, and final exam 

1.04 





Overall class grade 

Numerical course grade 

.38 

Meyer (1997) Study 1 

AL 

NRG 

Knowledge 

Open-ended 
questions exam 

Course posttest scores 

1.29 

Meyer (1997) Study 2 

AL 

NRG 

Knowledge 

Open-ended 
questions exam 

Course posttest scores 

1.17 

Miller et al. (2004) Study 1 

CH 

RCT 

Skills 

Motivational 

Overall MI Spirit 

.78 





Interviewing (MI) 
Skill Code (Miller 
& Mount, 2001) 

Percent MI consistent behaviors 

.83 

Miller et al. (2004) Study 2 

CH 

RCT 

Skills 

Motivational 

Overall MI Spirit 

.68 





Interviewing (MI) 
Skill Code (Miller 
& Mount, 2001) 

Percent MI consistent behaviors 

.59 

Miller et al. (2004) Study 3 

CH 

RCT 

Skills 

Motivational 

Overall MI Spirit 

1.12 





Interviewing (MI) 
Skill Code (Miller 
& Mount, 2001) 

Percent MI consistent behaviors 

.91 

Moreno-Montalvo (1987) 

AL 

RCT 

Knowledge 

Oral interview 

Language post-interview exam scores 

.71 





Observation 

Language skit presentation scores 

1.37 





Course content 

Course post test scores 

.36 
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Adult 

Learning 

Type of 

Outcome 

Type of 


Cohen’s d 

Study 

Method 3 

Study b 

Construct 

Measure 

Outcome Measures 

Effect Size 

Newsome & Tillman (1990) 

GD 

RCT 

Knowledge 

Course content 

Content knowledge exam 

.32 




Skills 

Application 

Accuracy of care plans 

1.89 






Simulated preparing a care plan 

1.25 

O’Connor & Korr (1996) 

CH 

NRG 

Self-efficacy 

Survey 

Self-efficacy 

.36 

Peterson (1996) 

AL 

RCT 

Knowledge 

Course content 

Application of concepts 

-.99 






Knowledge of cost management 
principles 

-1.10 






Comprehension of concepts 

-.01 




Attitude 

Rating scale 

Attitude about the learning method 

-.04 

Pierce & Miller (1994) 

CH 

RCT 

Skills 

Observational 

Effective teaching behaviors 

.03 





measure 



Portes et al. (1992) 

AL 

RCT 

Knowledge 

Course content 

Course final exam 

-.02 


AL 

RCT 

Attitude 

State-Trait 
Anxiety Inventory 
(Spielberger, 
Gorusch, & 
Lushene, 1970) 

Trait anxiety 

-.35 

Preziosi (1995) Study 1 

AL 

NRG 

Knowledge 

Course content 

Course posttest scores 

1.16 

Preziosi (1995) Study 2 

AL 

NRG 

Knowledge 

Course content 

Course posttest scores 

1.69 

Prichard (1990) 

AL 

RCT 

Knowledge 

Course content 

Course final exam (multiple-choice 
items test) 

.66 

Pugach & Johnson (1995) 

CH 

NRG 

Skills 

Frequency count 

Frequency of referrals to special 
education 

.47 




Self-efficacy 

Teacher Efficacy 
Scale (Gibson & 
Dembo, 1984) 

Personal and professional self-efficacy 

.48 





Survey 

Confidence 

.65 

Robinett (1976) Study 1 

AL 

RCT 

Knowledge 

Foreign language 
achievement 

Course posttest exam scores 

.64 




Attitude 

Frequency count 

Class absences 

.63 

Robinett (1976) Study 2 

AL 

RCT 

Knowledge 

Foreign language 
achievement 

Course posttest exam scores 

.85 




Attitude 

Frequency count 

Class absences 

-.18 

Schiffler (1986) Study 1 

AL 

RCT 

Knowledge 

Language exam 

Language vocabulary score at 7th yr 

.20 






Language structure score at 7th yr 

.00 






Language vocabulary score at 8th yr 

.00 






Language structure score at 8th yr 

.08 





Language C-Test 

Language proficiency 

.93 





(Schiffler, 1986) 

Language Translation from new 
language 

.97 






Language Translation into new 
language 

.55 






Language Oral communication 

.07 

Schiffler (1986) Study 2 

AL 

RCT 

Knowledge 

Language exam 

Language vocabulary score at 8th yr 

.00 






Language structure score at 8th yr 

.20 






Language vocabulary score at 9th yr 

.63 






Language structure score at 9th yr 

.61 
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Appendix C, continued 



Adult 

Learning 

Type of 

Outcome 

Type of 


Cohen’s d 

Study 

Method 3 

Study b 

Construct 

Measure 

Outcome Measures 

Effect Size 

Schiffler (1986) Study 2, 




Language C-Test 

Language proficiency 

.71 

continued 




(Schiffler, 1986) 

Language translation from new 
language 

.18 






Language translation into new 
language 

.49 






Language oral communication 

1.79 

Schuster (1976) 

AL 

RCT 

Knowledge 

Course content 

Oral language final exam scores 

.12 





exam 







Course content 

Written language final exam scores 

-.58 





exam 



Sears (1973) 

GD 

NRG 

Attitude 

Achievement 

Debilitating Anxiety 

.50 





Anxiety Test 
(Sears, 1973) 

Facilitating Anxiety 

.30 




Skills 

Course content 

Engineering Function Instrument exam 

.33 

Shaw (1980) 

GD 

RCT 

Attitude 

Frequency count 

Course drop-out rate 

1.07 

Showers (1982) 

CH 

RCT 

Skills 

Observational 
rating scale 

Transfer of training overall 

1.40 

Showers (1984) 

CH 

NRG 

Skills 

Observational 
rating scale 

Transfer of training overall 

1.80 





Teacher Innovator 

Factual information processing 

.04 





System (Showers, 
1984a) 

Conceptual information processing 

.50 





Theoretical information processing 

.34 

Simkins & Maier (2004) 

JIT 

RCT 

Knowledge 

Course content 

Exam scores 

1.27 

Slunt & Giancarlo (2004) 

JIT 

RCT 

Knowledge 

Frequency count 

Student taking chemistry pre-class 
quizzes 

.64 

Snyder (1980) 

GD 

RCT 

Knowledge 

Frequency count 

Course “A” Grades 

.72 

Sparks (1986) 

CH 

NRG 

Skills 

Stallings 

Teaching behavior and academic 

-.14 





Secondary 
Observation 
Instrument 
(Stallings, 1979) 

interactions 


Stahl etal. (1991) Study 1 

AL 

RCT 

Knowledge 

Basic Word 

Language immediate learning scores 

-.04 





Vocabulary Test 
(Stahl, Brozo, 
Smith, Henk, 

& Commander, 
1991) 

Language delayed recall scores 

.03 




Stahl et al. (1991) Study 2 

AL 

RCT 

Knowledge 

Basic Word 

Language immediate learning scores 

.11 





Vocabulary Test 
(Stahl etal., 1991) 

Language delayed recall scores 

.14 

Stein et al. (1982) 

AL 

RCT 

Knowledge 

Vocabulary word 

Language immediate vocabulary 

.47 





to definition 

retention exam 







Language 1 week delayed retention 

1.17 






exam 


Streufert (1985) 

CH 

NRG 

Skills 

Interpretive 

Teacher knowledge comprehension of 

.91 





Discussion 
Questionnaire 
(Streufert, 1985) 

specific reading program 






Observational 
measure (Hunter, 
1983) 

Teacher performance 

.85 
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Study 

Adult 

Learning 

Method 3 

Type of 
Study" 

Outcome 

Construct 

Type of 
Measure 

Outcome Measures 

Cohen’s d 
Effect Size 

Wynn (1987) 

CH 

RCT 

Skills 

Observational 
rating scale 
(Purdom, 1984) 

Purdom- Wynn Observation Instrument 
(PWOI): Introduction 
PWOI: Content presentation 
PWOI: Follow up/Feedback 
PWOI: management of student conduct 

1.17 

2.08 

.96 

.44 

Zeiss (1984) 

AL 

RCT 

Knowledge 

Test of English 
as a Foreign 
Language 
(Educational 
Testing Service, 
2008) 

Language abilities at 3 wks 

1.46 


a AL = Accelerated learning, CH = Coaching, GD = Guided design, and JIT = Just-in-time training. 

"RCT = Randomized controlled trial or NRG = Non-randomized comparison group study. 

'Outcome measures that include citations for the adult learning study indicate that the measure is included in the research report. 


Practical Evaluation Reports 


32 


Volume Two, Number One, March 2009 


